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Inside This Report 


In this report, we describe a project undertaken by the Community College Research 
Center (CCRC), Education Northwest (EdNW), and three community colleges in 
Oregon to adapt lesson study for use among faculty teaching a precollege (developmental) 
quantitative literacy course. Lesson study is a collaborative professional development 
approach that has been implemented among K-12 mathematics teachers with promising 
results for both teachers and students. There has been limited research on its application 
in higher education. We report findings from a mixed-method study of the adapted 
model’s implementation among faculty in the community colleges and report limited 
findings on student learning and progression in developmental mathematics. 


In the adapted lesson study model, a team of faculty members works to collaboratively 
design, teach, and reflect on the effects ofa lesson, defined as a one- to two-hour period of 
instruction focused ona specific topic or goal. During lesson study, team members engage 
in iterative cycles of inquiry and action. Each cycle consists of four stages intended to be 
completed in a single semester/term: (1) study and plan a lesson; (2) teach the lesson, 
collect observational data, and debrief the teaching; (3) revise the lesson, reteach witha 
different set of students, and collect observational data; and (4) reflect on the cycle results, 
document professional learning, and disseminate the findings to colleagues. 


The project activities unfolded across two phases. In the model development phase, 
faculty leaders were trained to implement lesson study over three cycles, during 
which the project team and faculty leaders collaborated on adapting the model for 

use in the community college context. Then, in the pilot study phase in fall 2019, the 
adapted model was implemented by faculty teams from the three colleges. A total of 
22 math faculty participated in the project. Across the development and pilot phases 
of the project, the research team used a mixed-method study design to examine how 
lesson study was adapted for community colleges, how community college faculty 
implemented the model, and how the model affected faculty and students. Data sources 
included interviews with participating faculty and college administrators, a survey of 
all mathematics faculty at the three colleges, observations of lesson study activities, 
an assessment of student learning, and administrative data that included information 
about students’ course enrollments and grades. This report includes the following 
project findings: 


e Lesson study is different from typical opportunities for professional 
development in community colleges. While lesson study shares some 
similarities with other collaborative and intensive professional development 
models such as faculty inquiry groups, it makes use of a highly structured approach 
to improving discipline-specific teaching practices. Protocols used in lesson study 
are distinctive in how they prompt faculty teams to observe and closely examine 
specific instructional strategies and their impact on student learning. 


e Faculty implemented lesson study with fidelity. Using a fidelity rubric 
developed in consultation with faculty leaders during the model development 
phase, the research team scored each faculty team during the pilot phase on 
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18 indicators of strong lesson study implementation. All four teams met the 
characteristics for either strong or moderate implementation for all 18 indicators. 


e Participating faculty were overwhelmingly satisfied with their lesson study 
experiences. The lesson study protocols invited the faculty team to identify 
collaboration norms, directed the team’s attention to long- and short-term 
learning goals, required the team to make collective decisions about detailed 
instructional plans, and provided the team with opportunities to closely 
observe student learning. These elements of the model fostered rich and detailed 
conversations about teaching and learning, which participants found valuable. 


e Faculty who participated in lesson study adopted new teaching practices. 
Analysis of observation notes and lesson plans from the model development and 
pilot phases shows that revised lessons included more open-ended, cognitively 
demanding tasks and new strategies to increase mathematical communication 
among students. 


e To sustain lesson study, community colleges must provide continued 
support. Lesson study requires significant administrative coordination and 
a skilled facilitator. One of the three colleges continued lesson study beyond 
the project period using funds from the department dean to provide faculty 
facilitators with released time from teaching and pay part-time faculty for their 
participation. Others intended to implement “light-touch” versions of lesson 
study that preserve important features of the model but require less time. 


e Due to limitations of the research design, we were largely unable to estimate 
the effects of lesson study on student outcomes. Students in some course 
sections taught by faculty who participated in lesson study performed better on 
some items in a researcher-designed assessment of student learning, as compared 
to students taught by nonparticipating faculty. We found no positive relationship 
between lesson study and students’ course grades or progression into college- 
level math. It may not be reasonable to expect substantial differences in student 
learning or academic outcomes after only one cycle of lesson study. What is more, 
we were unable to carry out a quasi-experimental estimate of the direct impact 
of even one cycle of lesson study on student outcomes because we did not have 
baseline data on students’ prior academic achievement. 
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Introduction 


Effective instruction is central to a range of ongoing reform efforts in higher education, 
including those that aim to increase online course offerings, address persistent student 
equity gaps, and improve the student experience by means of broad structural change 
(through, for example, guided pathways and developmental education redesign 
initiatives). However, much of this reform work does not sufficiently support faculty in 
strengthening their teaching practice through professional development. Professional 
development for higher education faculty typically takes the form of attendance at 
one-time workshops or conference sessions and is rarely hands-on. It often focuses 

on relatively decontextualized topics—such as using technology in the classroom or 
addressing common student learning disabilities—which may not address the specific 
instructional needs that faculty have within their particular disciplines. Moreover, 
part-time faculty, who make up two thirds of the instructional staff at community 
colleges, have less access than full-time faculty to the limited professional development 
opportunities that do exist. 


High-quality teaching is crucial for students taking developmental mathematics courses 
in community colleges, many of whom have been poorly served in their prior educational 
experiences. Succeeding in developmental mathematics can play a critical role in 
preparing these students for college-level coursework and getting 
them ona path toward college persistence, program completion, and, 


ultimately, upward mobility. Reform efforts are underway to redesign nlsh-quallty teaching 


. is crucial for students 
developmental course sequences at community colleges to better 


engage students and help them to succeed. Most of these efforts focus cart one tials 
on restructuring coursework, redesigning curricula, or enhancing in community colleges, 
student supports. However, as with reform efforts in higher education many of whom have 
more generally, few developmental mathematics reforms explicitly been poorly served in 
attend to student learning through improved pedagogy, which their prior educational 
could be facilitated through professional development for faculty experiences. 


(Edgecombe et al., 2013). 


Lesson study, a collaborative professional development model that focuses on 
improving instruction, has the potential to help faculty members make changes in 
their teaching practice that may improve student learning and success. In lesson study, 
small teams of faculty work in iterative “cycles” to collaboratively plan a lesson, observe 
the lesson’s implementation in a class with students, collect and analyze observational 
data on how students interact with the lesson materials, and revise the lesson based on 
their findings. Lesson study has been implemented among elementary and secondary 
mathematics teachers with promising results, but there has been limited application 
in higher education. In this report, we describe a collaborative project undertaken by 
the Community College Research Center (CCRC), Education Northwest (EANW), and 
three community colleges in Oregon to adapt lesson study for use in the community 
college context. We report on findings from a mixed-method study of the adapted 
model’s implementation and its effects on student learning and progression in 
developmental mathematics. 
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We found that, although lesson study is different from the typical professional 
development opportunities available in community colleges, faculty were able to 
implement lesson study successfully. Participants largely enjoyed their lesson study 
experience, particularly its focus on improving mathematics pedagogy and the 
opportunities it provided for intensive and productive collaboration. Faculty adopted new 
teaching practices as a result of their participation in lesson study. Owing to study design 
and time limitations, we were not able to rigorously evaluate lesson study’s impact on 
student outcomes Based on a descriptive analysis, we were able to detect only a limited 
positive relationship between faculty participation in lesson study and student learning and 
no positive relationship between faculty participation in lesson study and students’ grades 
or other academic progression milestones during the pilot study. However, drawing on our 
qualitative analysis, we present a theory of change for lesson study and offer suggestions 
for future research that could assess the impacts of lesson study on student learning and 
outcomes in community colleges. 


Rationale for Lesson Study in Community 
Colleges 


Within the field of professional development for educators, evidence from K-12 contexts 
points to a well-documented set of characteristics that produce changes in educator practice 
and improvements in student learning outcomes. These characteristics include professional 
learning opportunities that: are content-focused and of sustained duration, incorporate active 
learning for teachers, support collaboration among educators, use models of best practice, 
provide coaching and expert support, and offer time for feedback and reflection (Cohen & 
Hill, 1998; Garet et al., 2001; Kennedy, 1998). According to Darling-Hammond etal. (2017), 
effective professional learning incorporates most or all these features. 


In community colleges (and other higher education institutions), there are relatively few 
examples of professional learning opportunities with these characteristics. This is in part 
because the structure and culture of higher education have not typically fostered extended or 
intensive opportunities for professional development related to pedagogy (Furco & Moely, 
2012; McHenry etal., 2010). Long-held professional norms around faculty autonomy and 
limited instructional guidance serve as barriers to meaningful or sustained professional 
developmentin teaching. Faculty are not typically provided with opportunities to learn about 
evidence-based teaching practices and their application in the classroom. Moreover, faculty 
have an array of responsibilities that extend beyond teaching and therefore engage in many 
professional development activities that are not focused on pedagogy, including learning 
about institutional policies and structures, improving skills related to scholarship and 
research, and strengthening disciplinary expertise (Taylor & Colet, 2010). 


Evidence About Lesson Study 


Lesson study shows promise as an effective professional development approach for improving 
instruction. Although it originated in Japan, where conceptions and expectations of teaching 
and teachers are very different from those in the United States, it has been implemented 
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successfully in K-12 settings in the United States with research showing positive impacts 
on student learning. A 2014 literature review of 643 studies of professional development 
in mathematics found that lesson study was one of only two approaches with rigorous 
evidence of effectiveness in improving student outcomes (Gersten etal., 2014). A 
randomized controlled trial of lesson study among elementary school teachers showed 

a significant positive impact on students’ knowledge of fractions (Lewis & Perry, 2015). 
Ina study of lesson study involving high school geometry teachers, Barrett et al. (2013) 
found that students whose teachers participated in lesson study performed significantly 
better on benchmark exams than students whose teachers did not. 


These findings are not surprising, because lesson study aligns with the previously 
discussed research-based characteristics of high-quality professional development for 
educators. For example, lesson study groups teachers who teach common subject areas 
together, which has been found to increase collaboration and trust among teachers and 
build the capacity of teachers to learn together (e.g., Byrum et al., 2002; Garet et al., 2001; 
Wilms, 2003). The model uses active, hands-on approaches to teacher learning, with 
teachers planning lessons together, trying new teaching strategies, collecting data on 
their work, and reflecting on the results. The model also focuses on content-related topics, 
such as curriculum, content knowledge, and how students learn specific content, which 
has been found to be more effective in increasing teachers’ professional learning than 
focusing on general pedagogical approaches (e.g., Cohen & Hill, 1998; Kennedy, 1998). 
In many ways, lesson study produces a structured professional learning community, or 
PLC, an example ofa collaborative professional development model that incorporates 
several of the characteristics of effective professional development in K-12 settings that 
support student learning gains (Darling-Hammond etal., 2017). 


Although not rigorously studied, the use of lesson study in higher education has 

been documented (Cerbin, 2011; Cerbin & Kopp, 2006; Demir etal., 2012). A recent 
systematic review of lesson study among higher education faculty (Hervas, 2021) found 
five positive outcomes for faculty who participated in lesson study across 21 studies 
(listed here from most to least often reported): a shift in faculty practice from teaching to 
learning, a consequent shift to more interesting and/or meaningful teaching and learning 
experiences, increased collaboration and collegiality, other changes in faculty knowledge 
and practice, and an increase in continuing to engage in inquiry-based professional 
development and research. Significantly, however, all but one of the studies were limited 
to seven or fewer participants, and the results focused on impacts on faculty, not on 
outcomes for students. 


Lesson Study in the Developmental Math Context 


Many developmental mathematics courses at community colleges have undergone 
extensive structural and curricular redesign, but faculty may not have received support 

in instructional practice for these new courses. This is particularly concerning given 

the documented challenges associated with implementing effective instruction in 
reformed developmental mathematics contexts (Bickerstaff & Cormier, 2015; Cox, 
2018). For example, many community colleges have created quantitative literacy 
pathways, designed to shorten the developmental math sequence for non-STEM (science, 
technology, engineering, and math) students and prepare them for liberal arts math, a 
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college-level, transferable course. This newly redesigned developmental math pathway 
requires that faculty accustomed to teaching algebra develop a new body of pedagogical 
content knowledge that emphasizes mathematical reasoning about everyday mathematics 
concepts. Additionally, a significant proportion of developmental course sections in 
community colleges are taught by part-time or adjunct faculty, who are less likely to 
access on-campus supports and resources (Center for Community College Student 
Engagement, 2014). Thus, there is a need to identify models for instruction-focused 
professional learning that can be tested for usability in community colleges and for their 
promise in improving instruction and outcomes, particularly among those community 
college students who may be academically underprepared. 


Given its relatively strong research base in K-12 settings, lesson study appears well-suited 
for the community college developmental mathematics context. The model provides 

a clear structure for strengthening instruction and systematically examining student 
learning, which is necessary for the development and monitoring of specific instructional 
interventions. Once trained in lesson study, faculty can continue to deepen their 
expertise with the model through repeated implementation without the need for 
additional training. In this way, the initial training investment potentially pays high 
dividends in terms of faculty learning in the future. Furthermore, with its focus on 
collecting classroom-level data, lesson study is aligned with faculty inquiry models such 
as scholarship of teaching and learning (Boyer, 1990) and classroom assessment (Angelo 
& Cross, 1993), which have long histories in higher education. Finally, lesson study does 
not require faculty to enact wholesale course redesign—for example, to revise course 
goals, curricula, or assessments—unless they choose to; therefore, it may be perceived asa 
feasible approach in an environment that prioritizes faculty autonomy. 


Project Description 


CCRC partnered with EdNW and three community colleges in Oregon to implement 
lesson study in a precollege quantitative literacy course (Math 098). As part of a larger, 
statewide effort to redesign developmental education that began in 2014, community 
college math faculty developed Math 098, and nearly all community colleges in the 
state began to implement it in the 2014-15 academic year (Hodara & Petrokubi, 2017). 
In parallel, Oregon’s community colleges implemented other reforms to developmental 
education, including the integration of reading and writing skills into a combined 
course and the development of multiple measures for placement in developmental 
education courses. 


Math 098 provides Oregon community college students with an alternative, shorter math 
pathway to prepare for college-level math (Higher Education Coordinating Commission, 
2015). The course is designed for students whose degree or certificate goals do not 

require trigonometry or calculus and who are pursuing programs in liberal arts or other 
non-STEM fields. The course covers applied number sense, applied algebraic reasoning 
and modeling, graphical sense, measurement, and statistical reasoning. One of the 

three colleges also offers an additional prerequisite course (Math 058) in the non-STEM 
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pathway, which prepares students for Math 098. Faculty at that college conducted lesson 
study in both Math 058 and Math 098. 


Table 1. 


Partner College Characteristics 


FALL ENROLLMENT, PELL BLACK& GRADUATION 


DEGREE-SEEKING FULL-TIME GRANT HISPANIC RATE, 150% 
URBANICITY HEADCOUNT STUDENTS RECIPIENTS STUDENTS (3-YEAR) 


Clackamas Suburban 5187 43% 44% 15% 13% 
Community College 

Latics omima nity Rural 6,607 53% 50% 15% 19% 
College 

polenta Comma) Urban 24,404 42% 43% 17% 21% 


College 


Source. Academic year 2018-19, Integrated Postsecondary Education Data System (IPEDS). 


The project activities unfolded across two phases: (1) model development and (2) a 

pilot cycle and study. During the model development phase, lasting from spring 2018 
to summer 2019, each community college identified a leadership team comprising 
about four full-time and part-time mathematics faculty members with an interest in 
refining instruction in Math 098. EdN W facilitators provided these teams with initial 
training on lesson study, and then the leadership teams conducted three cycles of lesson 
study facilitated by EdN W staff. After each cycle, faculty provided feedback on their 
experiences to EANW, which informed the refinement of the adapted model and the 
development of facilitation materials. The leadership teams from the three colleges 
convened all together three times during this phase to deepen their understanding of 
research-based instructional practices, share what they were learning about instruction 
as a result of participation in lesson study, build consensus on the components of the 
adapted model, provide feedback on draft facilitation materials, and discuss strategies for 
sustaining lesson study at their colleges. (For more information about project activities, 
see Bickerstaff et al., 2019.) 


In fall 2019, between one and three members of each leadership team volunteered to 
serve as facilitators ofa pilot lesson study cycle with colleagues from their department 
who had not yet been involved in lesson study. Four teams at the three colleges 
participated in this pilot of the adapted lesson study model. The teams used lesson study 
materials developed by EANW, but EdN W facilitators did not participate in the pilot 
implementation. 


A total of 22 math faculty participated in the project. Compared to the broader population 
of faculty teaching developmental mathematics at the three colleges, faculty who 
volunteered to participate in lesson study were different from their colleagues in that 

they were more likely to be female, less likely to be White, less likely to have a doctorate, 
and slightly younger with less college teaching experience (as described in Table 2). These 
differences may be attributable to the fact that lesson study was focused on Math 098, a 
new quantitative literacy course that faculty largely self-selected to teach. 
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Table 2. 


Characteristics of Faculty Participants and Nonparticipants 


LESSON STUDY OTHER DEVELOPMENTAL 


FACULTY CHARACTERISTICS PARTICIPANTS EDUCATION MATH FACULTY 
Employment status 
Full-time faculty 36% 36% 
Part-time or adjunct faculty 64% 60% 
Other 7 4% 
Gender 
Male 23% 40% 
Female 64% 52% 
Other/no response 14% 8% 
Age (years) 475 49.9 
College teaching experience (years) 24.9 26.6 


Highest degree 


Doctorate or professional 5% 12% 
Master's 82% 86% 
Bachelor's 14% 2% 


Race/ethnicity 


White 73% 82% 
Black 2 2 
Asian 9% 6% 
Hispanic 5% - 
Middle Eastern = 2% 
Other/no response 18% 16% 
N 22 50 


Across the model development and pilot phases of the project, the research team used a 
mixed-method study design to address four research questions related to the adaptation and 
implementation of lesson study and the influence of the model on instructors and students 
(see Table 3). 
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Table 3. 


Research Questions and Data Sources 


RESEARCH QUESTIONS DATA SOURCES 


Adaptation and implementation 


1. Howcan lesson study be adapted for implementation in e Observations of training activities, convenings, and lesson 
acommunity college context and what are the adapted study cycles 
model's core components? e Interviews with EdNW facilitators and faculty leadership 
teams 
2. Does lesson study influence instructors’ beliefs, curricular e Observations of lesson study cycles across both project 
materials, and teaching practices? phases 


e Review of lesson plans 
e Faculty survey 
e Interviews with faculty 


3. Can lesson study be feasibly implemented in higher e Interviews with faculty and college administrators 
education? 


Student outcomes 


4. What is the relationship between faculty participation in e Assessment of student learning 
lesson study and student learning outcomes? e Student-level administrative data 


To address the first three research questions focused on adaptation and implementation, 
we collected several forms of qualitative data. To capture the perspectives of faculty 
participants and to understand the feasibility of implementing lesson study, we conducted 
interviews with members of the leadership teams at the beginning and end of the 

project period and interviews with other stakeholders in the interim, including college 
administrators and faculty who participated in the pilot study. We conducted a survey of 
all math faculty at all three colleges in fall 2018 and fall 2019. The research team observed 
and took field notes at training and adaptation activities as well as at the lesson study 
cycles during the model development and pilot study phases. For each lesson study cycle, 
the research team collected two versions of the lesson plan; in cases where the team was 
working from a preexisting lesson, researchers also documented a third baseline version 
of the lesson plan. Through our implementation research, we sought to understand 
faculty experiences with lesson study, the ways in which the approach is different from 
other typical professional development opportunities, and faculty satisfaction with their 
experiences. We also identified lessons that could be useful for other postsecondary 
institutions interested in implementing lesson study. This research also informed 
adaptations to the lesson study model during the model development phase. 


To document the effects of lesson study on students, the research team and members of 
the faculty leadership teams codeveloped an assessment of learning on percentages, a key 
concept in the quantitative literacy pathway, which we administered in 2018 and 2019, 
before and after the lesson study pilot implementation. Additionally, we collected and 
analyzed student-level administrative data from each college that included information 
on students’ course enrollments and course grades. These data sources and our analytic 
approaches are discussed in further detail in the findings sections below and in the 
Appendices (available in a separate document). 
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Model Development: Adapting Lesson 
Study for Community College Faculty 


Overview of the Adapted Lesson Study Model 


The model of lesson study that was refined during the model development phase and 
used in the pilot phase of this project is adapted from the version of lesson study described 
in Leading Lesson Study: A Practical Guide for Teachers and Facilitators (Stepanek et al., 
2007). In the adapted model, teams of four to five faculty members work in iterative 
inquiry cycles to collaboratively design, teach, and reflect on the effects of a lesson— 
which, in the community college context, is defined as a one- to two-hour period of 
instruction focused ona specific topic or goal. Each team is guided by a self-selected 
research theme that is intended to establish the direction for at least one year of lesson 
study work. Typically, the research theme connects to a fundamental purpose of the 
course, defines a problem of practice that faculty participants care deeply about, and 
relates to long-term goals for students. For example, one team in this project selected the 
research theme: “How do we build students’ confidence in their mathematical reasoning 
and willingness to persevere in problem-solving?” 


During lesson study, team members engage ina cycle of inquiry and action. Each cycle 
consists of four stages intended to be completed in a single semester/term (see Figure 

1). First, the team sets its research theme, examines the course curriculum, identifies a 
topic that is challenging to teach or a lesson they want to improve, and develops a detailed 
plan tied to a set of learning goals for a class period. Second, one team member teaches 

the lesson while others observe students and record evidence of learning; together, the 
team debriefs the experience. Third, the team draws on observation data to revise the 
lesson plan, and a team member reteaches the revised lesson to a different set of students. 
Finally, the team reflects on the cycle results, documents its learning, and disseminates 
the findings to peers and other stakeholders. 


LESSON STUDY IN COMMUNITY COLLEGE MATHEMATICS | OCTOBER 2021 EEE 


Figure 1. 
Lesson Study Cycle Stages 


Study & Plan 1 
The team identifies goals for students, 
investigates curricula, and examines 
research on classroom practice and 


student learning. Then team members =, 2 | 
e 


collaboratively develop a detailed lesson plan 


that includes tasks, anticipated responses, e ome Teach, peste, 
instructor moves, and evaluation questions. (iy (a) & Debrief 
ae) One team member teaches the 


lesson while others observe 
and record evidence of student 
learning. Next, team members 
share observation data, 

4 | discuss evidence of student 


learning, and explore whether 


students achieved goals. 
Reflect & Report 


The team reflects on the results of the cycle 
and documents its learning in a report. 

The team also plans how it will share and Revise & Reteach 
disseminate its new professional knowlege 
with peers and other interested stakeholders. 


. 
if 


The team uses its observation findings to revise the 
lesson for improved effectiveness. A team member 
reteaches the lesson while other team members observe 
and collect evidence of student learning. Then the team 
debriefs the reteaching and explores how changes in the 
lesson may have infuenced student learning outcomes. 


Each of these stages is guided by a set of structured protocols that encourage faculty members 
to investigate evidence-based practices, draw connections between instructional decisions 
and the research theme, and consider what students do and do not understand. The full 
cycle, including meetings, classroom observations, and preparatory work, takes up to 20 
hours. The team must have access to two sections of the focal course so that the lesson can be 
retaught with a new set of students.’ 


The Process of Adapting the Lesson Study Model 


The model described above was developed by the leadership teams and EdN W facilitators 
during the first phase of the project, as a major project goal was to adapt the original model 

of lesson study to create one that is usable and feasible at community colleges. Many of 

the revisions made to the original model and accompanying facilitation materials were 

in response to the need fora streamlined process that is sensitive to time constraints and 
provides sufficient guidance for higher education faculty, most of whom are content experts 
in mathematics rather than trained educators. Templates to guide a team’s planning decisions 
and creation ofa lesson plan were simplified and scaffolded to support faculty members who 
may have limited experience collaborating with colleagues on instruction. For example, 
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because leadership teams found it challenging to prioritize what to change from the first 
teaching of the lesson to the second and tended to want to revamp a significant portion of 
the lesson, EAN W facilitators refined the revision process and protocol. The new process 
provided an opportunity for individuals to review the lesson plan and underline sections 
they wanted to keep and sections they wanted to change, informed by observations of 
student learning. This refinement resulted in a more focused revision process that was 
intentional and efficient. While some elements of the original model were eliminated 

for greater simplicity, the model retains essential features critical to lesson study, such as 
crafting goals, exploring instructional materials, analyzing tasks, anticipating student 
responses and misconceptions, and outlining pedagogical decisions. 


Recognizing the scheduling challenges associated with implementing the model in 
community colleges, EAN W facilitators and the leadership teams discussed the option 

of not including stage 3, Revise and Reteach. Some existing lesson study models do 

not include this phase, and coordinating the logistics of a reteach presents significant 
challenges. Teams must have access to two course sections, and they must coordinate 
teaching schedules across course sections so that the teaching, revision, and reteach 

stages work with the natural flow of both courses. (In the lesson study cycles during the 
model development and pilot phases, most teams elected to first teach the lesson ona 
Wednesday or Thursday, revise on Friday, and reteach in the second section on Monday or 
Tuesday of the following week.) Despite these challenges, faculty found the learning from 
reteach too important to skip in this project. 


As part of the model development phase, we drew on observational data, interviews, 
and feedback from faculty to identify the core components of the adapted model. Based 
on these core components, and with input from faculty leaders, we outlined a set of 

18 indicators of strong lesson study implementation, which the research team used to 
measure fidelity of implementation during the pilot (see Appendix B). Among these 
indicators are three implementation practices identified during the model development 
phase (Bickerstaff et al., 2019), which reflect the behaviors and patterns of engagement 
that may help faculty realize the benefits of lesson study in the community college 
context (see Figure 2). The first implementation practice is to develop and sustain a 
collaborative lesson study team. This practice points to the importance of building trust 
among team members and developing a collective belief that gaps in students’ knowledge, 
understanding, and performance can be remedied, at least in part, by an inquiry-focused 
approach to improvement. The lesson study materials include processes to build each 
team’s capacity for collaboration, including establishing a clear purpose for lesson study 
through the research theme, developing and abiding by team collaboration norms, 

and maintaining an inquiry focus on student learning (rather than faculty evaluation) 
throughout the process. 


The second implementation practice is to study research and apply evidence-based 
practices. Without this emphasis, lesson study participants may design and refine lessons 
in ways that are counter to the best available evidence on student learning. The focus on 
research on instruction allows faculty to translate empirical evidence into classroom 
practice. The third implementation practice is to generate and share professional 
knowledge. This practice aims to broaden the influence of lesson study by inviting 
nonparticipating faculty to improve their instruction based on the learnings of the lesson 
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study team. This may happen through the team disseminating refined lesson plans or 
sharing a broader set of instructional strategies uncovered during their experiences in the 
lesson study cycle. This practice also prompts teams to reflect on generalizable lessons 
learned from the cycle and the ways in which the lesson study experience can have 


longer-term and more far-reaching impact beyond a single class period. 


Figure 2. 


Lesson Study Implementation Practices 
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Comparing Lesson Study to Typical Professional 
Development in Community Colleges 


To understand the extent to which lesson study is distinct from other opportunities 

for professional learning for faculty at community colleges, we asked faculty and 
administrators about typical professional development opportunities available to 
full-time and part-time mathematics faculty. Almost 90% of developmental mathematics 
faculty in our baseline survey (fall 2018) reported receiving some form of professional 
development in the past year. In interviews, most faculty reported access to professional 
development funds, which can be applied to conference attendance or tuition 
reimbursement, and all faculty and administrators described workshops, speakers, 

and events hosted by the college (often in the context ofa staff in-service day). Most 
on-campus professional development offerings are relatively low intensity, in the form of 
a one-hour workshop or one in-service day. And most professional learning experiences 
are pursued by individual faculty members rather than by groups of faculty together. 


The adapted model of lesson study, which is intensive and collaborative, is different 

from typical professional learning opportunities available to community college faculty. 
A minority of faculty reported participating in professional learning experiences with 
similar features to the lesson study model. At baseline, 26% of faculty survey respondents 
reported receiving more than 15 hours of professional development in the previous year. 
Examples of more time-intensive learning experiences include graduate-level coursework 
and multiday professional conferences. Among survey respondents who participated in 
any professional development in the year prior to the start of the project, 41% reported 
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participating in professional development that was collaborative in nature. For example, 
several faculty we interviewed had participated in faculty inquiry groups that convened 
faculty with common interests to explore a topic collaboratively over the course ofa 
semester or academic year. 


While the adapted lesson study model shares some similarities with faculty inquiry 
groups and other collaborative structures in higher education like scholarship of teaching 
and learning, itis distinct from other available professional learning opportunities in 

its highly structured approach to improving disciplinary teaching practices embedded 

in a specific course. When asked about on-campus professional development offerings, 
interview respondents described a range of cross-disciplinary topics including 
instructional technology, accessibility of course materials, active learning strategies, 
culturally responsive teaching practices, and institutional initiatives and priorities. For 
instance, two colleges offered a teaching squares model, in which four faculty observe one 
another’s courses and then meet for reflection. This is intended to be cross-disciplinary 
to expose faculty to different teaching approaches beyond their department. The vast 
majority of survey respondents (82%) reported participating in fewer than 10 hours of 
discipline-specific professional development during the previous year, and only 29% 
reported participating in a learning experience that involved observing their colleagues’ 
teaching at least oncea year. 


The structure of lesson study prompts teams to observe and closely The structure of lesson 
examine specific instructional strategies and their impact on study prompts teams 
student learning. Some faculty reported that, prior to participating to observe and closely 
in lesson study, they sought out ad hoc and informal venues to ones eset . 
strengthen their mathematics teaching. For example, several metuctone strategies 
' : : : : and their impact on 
interviewees described arranging to observe colleagues in advance ‘ 

student learning. 


of teaching a new course. Others described informal reading groups 
or collaborative relationships that emerged from shared interests. 
By contrast, through its focused goals and protocols, the lesson study model provides 
a degree of structure that is lacking in these informal approaches to strengthening 
teaching practices. 


Table 4. 


Comparing Lesson Study to Typical Community College Faculty Professional Development 


TYPICAL PROFESSIONAL DEVELOPMENT ADAPTED LESSON STUDY MODEL 


Cultivated among the team, 


Expertise External speakers or off-site conference Withikiale trom researoh 
Teaching Focus Often discipline-agnostic Content-specific 

Intensity One-day workshop or shorter 15-20 hours per cycle 
Collaboration No, pursued by an individual Yes, pursued by ateam 
Structure Limited guidance for practice Detailediprotacals fornalatainfoous on 


teaching and learning 
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Findings on Lesson Study 
Implementation Among Faculty 


Once the model development phase was complete, in fall 2019, four teams of faculty from 
the three colleges participated in a lesson study cycle facilitated by members of the faculty 
leadership teams, using materials developed by EAN W. The pilot included both faculty 
who had participated in the model development phase and faculty who were new to 
lesson study. Faculty volunteered to participate in the pilot. Our findings on how faculty 
implemented the adapted lesson study model, their satisfaction with the model, and the 
model’s effects on their teaching practices are presented below. 


Implementing Lesson Study With Fidelity 


Members of the research team observed all team meetings and classroom observations 

of all four teams during the pilot phase. Using a fidelity rubric that was developed in 
consultation with faculty leaders, the research team scored each team on the 18 indicators 
of strong lesson study implementation described above using a three-point scale. A score 
of “1” means the indicator was not present, “2” reflects moderate implementation, and 
“3” reflects strong implementation (see Appendix A). 


All four teams met the characteristics for either strong or moderate implementation for all 
18 indicators. Only four indicators had an average score of less than 2.5 (meaning three or 
more teams received a moderate implementation score for those indicators ). For two of 
these four indicators, the moderate implementation was a reflection of lack of time. The 
lesson study protocol prompts teams to generate anticipated student responses and points 
of evaluation to prepare for the classroom observation. Teams were likely to run out of 
time during planning meetings and therefore took a more cursory approach to these 
lesson planning tasks. 


The two additional indicators with moderate scores point to potential challenges to 
implementing the adapted lesson study model at community colleges that go beyond 
time constraints per se. One relates to applying evidence-based instructional approaches. 
During the lesson study cycles in the model development phase, EdN W facilitators 
shared research literature on mathematics teaching and learning, and teams used these 
resources in their lesson planning. For example, EdN W facilitators provided research 

on strategies to increase the cognitive demand of mathematics tasks (i.e., Smith & Stein, 
1998), and teams incorporated more cognitively demanding tasks in their lessons (this 
will be discussed further below). In contrast to the model development phase, during the 
pilot phase, teams were responsible for finding their own research. Perhaps asa result, 
teams were likely to reference evidence-based practice in general terms, but they were less 
likely to actively engage in discussions of research articles or draw explicit connections 
between their instructional decisions and literature on mathematics education. Faculty 
participants indicated that incorporating research into lesson planning was logistically 
challenging with the timing of the cycle but also difficult because team members were not 
necessarily knowledgeable about relevant literature. One participant explained: 
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One thing that we just really didn’t do was to seek out research or articles 
that would help us to inform improvements to make in the second 
teaching. And part of that was just because to get it all on our calendars 
in a way that would work for everybody—we just didn’t have the time. 
And digging around and finding research is not something that we’re 
particularly good at because we don’t do it all the time. And it was just 
really hard for us to find something applicable, get it out to everybody, 
and give people time to read it over for us to talk about it. 


The final indicator with a moderate implementation score was sharing knowledge, which 

reflects one of the goals of the final stage of lesson study in which faculty reflect on what 

they learned during the cycle and make plans to disseminate that learning to others. 

During the pilot cycle, each faculty team discussed possible venues for dissemination, but 

they made few concrete plans. In interviews, faculty reflected on the opportunities and 

challenges related to sharing knowledge from lesson study. Many noted that there were 

few formal venues for sharing information about teaching practices 

within their departments. Each team concluded its lesson study In postsecondary 
cycle with a revised lesson plan, but it was not always clear if and education, faculty are 
how those plans should be shared. Two of the three colleges used unaccustomed to sharing 
an Open Educational Resource (OER) as the curricular material lesson plans. 

for Math 098. The teams at these colleges edited the OER based 

on knowledge gained from their lesson study cycles, but because the OER is a textbook 

for students, it was not seen as an adequate venue to share all of the learning associated 

with the cycles. In postsecondary education, faculty are unaccustomed to sharing lesson 

plans, and detailed lesson plans developed as part of lesson study were seen as particularly 
unfamiliar to postsecondary faculty who did not participate in lesson study. One 

participant explained: 


The lesson plan that [we] wrote was like ten pages long. You can’t give that 

to somebody and be like, “Okay, you need to teach this tomorrow.” So that 
is the real big challenge we have with lesson study—it’s very, very hard to 

share what you learn unless you're participating. 


Participants at the third college, which used a published textbook rather than an OER, 
also conveyed concern that colleagues may be less likely to use their revised lessons if they 
were perceived as misaligned with the textbook. 


The four indicators with comparatively weaker implementation scores have implications 
for designinga lesson study program that is sustainable at community colleges, which 
will be discussed in more detail below. But despite these difficulties, teams generally used 
the materials and practices associated with the adapted model effectively. 


Collaborating to Improve Teaching and Learning 


Participants in the pilot phase were overwhelmingly satisfied with their lesson study 
experiences. Ninety-five percent of faculty indicated in the survey that they were “very 
likely” or “somewhat likely” to participate again ifadditional lesson study cycles were 
available at their college. A large majority of participants also indicated that lesson study 
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improved their curricular materials, helped build their professional community, and 
developed their understanding of how students think and learn (see Figure 3). 


Figure 3. 
Participants’ Responses About Their Perceptions of Lesson Study (N = 22) 
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Data collected during the model development phase of the project suggest that the 
benefits to lesson study seem to accrue after multiple cycles. The first cycle is marked by 

a significant learning curve as instructors become familiar with the stages in the process, 
which are unlike other aspects of faculty work. In subsequent cycles, faculty reported 
worrying less about navigating the steps of the process and focusing more on their goals to 
improve teaching and student learning.” 


Lesson study’s distinctive features—the intensive collaboration, focus on disciplinary 
teaching, and prescriptive structures and protocols—contributed to faculty’s satisfaction 
with the model. In interviews, faculty described a desire for more opportunities 

to collaborate with colleagues and for more collaborative professional learning 
opportunities. One participant, who previously taught in K-12, described her experience 
in both contexts: 


I just value so much the idea of collaboration, and I don’t think we get it 
enough in community college. We get it a lot in public school. I feel like it’s 
part of the culture of public school, to have professional development as 
part of your job. 


Another participant summarized her experience with lesson study, highlighting what’s 
gained from the collaborative process: 


I think the biggest plus, in my opinion, is being able to get in other people's 
classrooms and observe them. To see their students, hear them talk, interact 
with one another. And then getting together as a team to discuss that. 
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While professional development was certainly available to faculty prior to the 
implementation of lesson study, the typical opportunities (conferences and workshops) 
did not yield the type of deep collaborative engagement that many faculty members 
valued. One faculty member described departmental colleagues as an untapped resource 
for peer learning: 


Professional development means taking your contracted [professional 
development] funds and going on a plane somewhere and going to a 
conference. And that’s the way we think about it. So it’s a very foreign idea 
to propose that we spend that money on ourselves paying for our time to 
talk to each other and tap into the deep wells of expertise and care that we 
have on our campuses. 


This respondent and others argued that the funding that is typically allocated to conference 
travel would be better spent releasing faculty from some teaching responsibilities so they 
are able to participate in collaborative learning experiences like lesson study. The time 
intensity of lesson study, up to 20 hours for a cycle in the adapted model, posed workload 
and logistical challenges (discussed further below), but faculty indicated that the depth 

of their engagement was more productive than lighter-touch professional learning 
opportunities. One faculty member reflected on the limitations of a half-day workshop: 


Three hours is not really enough to change the way you do business. It’s just 
enough to know that that’s maybe something I should look at. And so that is 
a problem. I’ve been to a couple trainings the past couple of years, and you 
come out of them feeling like, “Okay, but I still don’t know what to do.” 


Unlike other collaborative professional learning experiences (e.g., faculty inquiry 
groups), lesson study is highly structured, with protocols and guidance for each stage of 
the cycle. For an example of these structured protocols, see Figure 4. During their first 
cycle, many faculty participants found this structure uncomfortable and foreign, but 
those who participated in multiple cycles over the development and pilot phases came 
to appreciate the ways it helped facilitate productive conversations about teaching and 
learning. One college had previously brought together faculty teaching the same course 
for several meetings within a semester. Several stakeholders described challenges with 
that approach, including finding discussion topics that would engage both novice and 
experienced instructors. One faculty member at that college reflected: 


That part has been missing—like, looking actually at student work or 
talking about how students are engaging in this deep way. We did talk 
about how students are going to have trouble with this type of problem, 
but why? Why are they having trouble? We never really had a method to 
go deep into that. 


A faculty member from a different college spoke more generally about how instructors lack 
experience in conversing deeply with each other about instruction and student learning: 


We consistently find it challenging to talk to one another in a meaningful 
way. Not just, “Oh, here’s what we should put on the test. This is how 
many students got it wrong.” We want to do something that’s richer. 
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How do our students know things? What do I look for to understand 
their knowing? How do I open the door for them to come in toward this 
concept? Those kinds of conversations. 


Another faculty member explained how the protocols in lesson study created an 
environment that was conducive to experimentation and risk-taking. 


Because of the [debriefing] protocol, there’s no finger-pointing. Everybody 
developed this lesson and everybody thought that this is the way that 
things should go, so you're off the hook with that. You’re taking a risk by 
having people [observe you], but at the same time, it’s not. I think that a 
great thing about this is that you can kind of experiment with teaching 
without it necessarily being about judgment. 


Figure 4. 


Example of How Lesson Study Provides Structure for Faculty Collaboration 


The lesson study materials provide detailed guidance for faculty to engage in conversations about 
teaching and student learning. Here we present an excerpt from the debriefing protocol that takes 
place after the observed teaching session, providing guidance for how to share feedback and discuss 
the session. 


e Introductions (5 minutes). The facilitator expresses appreciation to the instructor for welcoming 
observers into their classroom and to the team for their work on the lesson. The facilitator briefly 
restates the lesson goals, learning outcomes, and research theme. 


e Teacher/Instructor Reflections (5 minutes). The team member who taught the lesson shares 
their thoughts about implementing the team’s plan, including both successes and challenges. 
This team member leads the way, giving everyone permission to genuinely analyze the lesson and 
offer feedback based on the evidence. The teacher/instructor’s initial observations will set the 
stage for others to take an analytic but supportive approach to the discussion. 


e Sharing Observational Data (15 minutes). Lesson study team members, followed by other 
observers, present data from the lesson focusing on evidence of student thinking and the 
questions and evaluation points noted in the lesson plan. Comments should focus on what was 
seen and heard and avoid subjective statements. In addition to anticipated student responses, 
were there any that were unanticipated? 


e General Discussion (15 minutes). The facilitator invites a more free-flowing discussion among 
team members and observers. Additional questions can be asked or observations shared; 
comments already offered can be probed at a deeper level; and ideas for strengthening the 
lesson can be proposed. 


The lesson study protocols invite the faculty team to identify collaboration norms, 
direct the team’s attention to long- and short-term learning goals, require the team to 
make collective decisions about detailed instructional plans, and provide the team with 
opportunities to closely observe student learning. These elements of the model fostered 
rich and detailed conversations about teaching and learning. 
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Adopting New Instructional Practices 


Almost three fourths of lesson study participants reported that their teaching practices 
changed because of their involvement in lesson study (see Figure 5). To explore how their 
instructional habits changed, we analyzed observation notes and lesson plans from 10 lesson 
study cycles conducted during the development and pilot phases looking for differences 
between baseline lessons (i.e., how lessons were taught before lesson study) and the lessons 
used in the second observation (i.e., the reteach). We identified two features that changed: 

(1) revised lessons included more open-ended, cognitively demanding tasks, and (2) 

revised lessons included new strategies to increase mathematical communication among 
students. Cognitively demanding tasks have multiple solution pathways; ask students to 
draw connections between multiple representations (e.g., visual diagrams, manipulatives, 
symbols, problem situation) and between and among mathematical ideas in novel situations; 
and ask students to “engage with conceptual ideas that underlie the procedures” (Stein et 

al., 2000, p. 16). Mathematical communication refers to conversations among students that 
focus on possible approaches to solving a problem and justification for those approaches 
(Imm & Stylianou, 2012; Pourdavood & Wachira, 2015). 


At baseline, the curricular materials for Math 098 at all three colleges contained many 
problems that were contextualized in real-world scenarios. At the same time, the problems 
were not always cognitively demanding. Instead, student thinking was frequently guided 
through a series of fixed tasks that emphasized procedures. For example, the following set 
of questions referred to a bar graph showing the number of earthquakes in a region by year: 


a. What was the change in the number of earthquakes from 2009 to 2010? 
. What about the change from 2014 to 2015? 

c. Which of the changes in problems (a) and (b) was the greatest? How do 
you know? 

d. Use your answer in problem (a) to find the percent change by computing 
the amount of change compared to 2009. 

e. Use your answer in problem (b) to find the percent change compared to 
2014. 

f- Doyour answers to (d) and (e) tell you anything new about these two 
changes? 


At baseline, most faculty who participated in the lesson study project used the majority 
of their class time to have students work in small groups or pairs. Yet the nature of the 
curricular materials meant that there were limited opportunities for rich mathematical 
discourse. Instead, in group settings, students often worked on problems on their own 
and then compared answers with their partners. These comparisons were not typically 
accompanied by discussions about why or how students arrived at these answers. 


As compared to the baseline lessons, lessons designed in the lesson study cycles provided 
less scaffolding, sometimes included problems that did not have clear, correct answers, and 
frequently included problems that had several possible solution approaches. For example, 
one team organized a lesson around making sense of this statement: “The wealthiest 

20% of the population have 90% of the wealth.” Students were asked to engage in several 
activities without clear, correct answers, including organizing 100 pennies (representing 
100% of the wealth) into quintiles. 
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In other examples of revised lessons, faculty increased the cognitive demand of new tasks 
by allowing students to explore the procedures with other students before explicitly 
teaching an algorithm. For example, at the start of a lesson on unit rates, students were 
asked to work together in groups to use their prior knowledge, estimation skills, and 
numerical reasoning to solve the following problem: 


My friend and I were at a gas station. I bought a premium 16.9-ounce 
bottle of water for $2.49. My friend bought 8.75 gallons of gas for $28.26. 
Compare the price of the gas per gallon to the price of water per gallon. 


This task is cognitively demanding because students were not given a procedure to follow and 
therefore needed to draw on their numeracy skills to develop a reasonable approach to solve 
the problem. Observations from the class sessions in which this lesson was taught showed 
that different groups used different solution approaches to arrive at the correct answer. 


In revised lessons, faculty also introduced new instructional strategies to facilitate 
mathematical conversations among students, a strategy that complemented the more 
cognitively demanding tasks. One team adopted a gallery walk format that they used in 
several lesson study cycles. Small groups of students were presented with a task without a 
prescribed solution approach. The group was asked to represent their thinking on a large piece 
of chart paper. Once all of the groups posted their work, each member of the group reviewed 
one other team’s chart paper. Then they returned to their team to discuss these prompts: 


What is similar to your team’s approach? What is different than your team’s 
approach? On a post-it note, write down something you like about their 
approach or a question you have. After looking at all the posters, do you think 
your team’s strategy and result are reasonable? Why or why not? 


Other lesson study teams employed different strategies to enhance mathematical 
communication. For example, one team asked students to write their work on note cards, 
which the instructor collected and showed anonymously on the document camera so that 
students could describe what they observed. This allowed the instructor to select specific 
approaches to be analyzed by the class. In another example, two small groups of students 
were given the same problem and, after solving it, were asked to combine into a single larger 
group to explain their reasoning. 


Increasing cognitive demand and providing opportunities for mathematical communication 
are aligned with evidence-based approaches to improve mathematics instruction (Imm 

& Stylianou, 2012; Pourdavood & Wachira, 2015; Stein etal., 2000). We used faculty 
survey data to understand more about faculty beliefs related to these and other instructional 
practices.* While we cannot match respondent data at baseline and follow-up to measure 
whether lesson study changed the attitudes of individuals, we can compare lesson study 
participants’ results with those of two groups: all developmental mathematics faculty at 
baseline and nonparticipating faculty at follow-up (see Figure 5). We do see some differences 
that reflect the types of changes in instructional practices that we documented during the 
lesson study cycles. Compared to nonparticipants, lesson study faculty were less likely to 
agree with the statement, “Instruction should be built around ideas and problems with clear, 
correct answers.” Compared with both nonparticipants and all baseline respondents, lesson 
study participants were more likely to agree with the statement, “Thinking and reasoning 
processes are more important than specific curricular content.” 
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Figure 5. 
Percentage of Faculty Who Agreed or Strongly Agreed With Statements About Instruction 
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My role as an instructor 
is to facilitate 
students’ own inquiry 


Instruction should be built 
around ideas and problems 
with clear, correct answers 


| prefer using familiar 
teaching methods over 
trying new approaches 


Cooperative learning is 
consistent with my 
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Note. We conducted pairwise statistical testing of means between baseline 2018 survey respondents (all developmental math faculty) 
and lessons study participants and between 2019 follow-up survey respondents (nonparticipating developmental math faculty) and 
lesson study participants. 


** significant at the 5% level, * significant at the 10% level. 


22 


LESSON STUDY IN COMMUNITY COLLEGE MATHEMATICS | OCTOBER 2021 EEE 


When asked if they planned on incorporating these new practices into their teaching 
beyond the pilot study cycle, lesson study participants reported that they did. The largely 
positive reaction from students to the lessons faculty implemented during the pilot cycle 
provided encouragement for faculty to retain these approaches as part of their ongoing 
teaching practice. Our data include numerous examples of participants describing 
students’ engagement with the lessons they designed: 


The most direct thing I learned was just how much more engagement we 
could get, just having students work together on a chart paper. All of a 
sudden, the energy felt different than having to put [the answer] up on the 
board, where sometimes students can be reluctant. 


These positive experiences influenced instructors’ perceptions about students’ ability to 
persevere when given more open-ended tasks. As one participant explained, “[I can] let 
them be in that confused space longer. I don’t feel like I have to immediately pull them out 
of that.” Lesson study supported faculty to develop “eyes to see students” (Lewis, 2002); 
it prompted them to focus their attention on students and their processes for learning. 


To Sustain Lesson Study, Institutions Must Provide 
Continued Support 


Most faculty who participated in this project enjoyed their lesson study experience, and 
many adopted new evidence-based instructional practices as a result of their participation. 
At the same time, most of the faculty who we interviewed for this study expressed concerns 
about whether lesson study would be sustainable at their college without continuous 
institutional investment. During the project period, grant funds were used to provide 
stipends or payment to part-time faculty for their participation, and some colleges also 
provided support to some full-time faculty who took on leadership 

roles during the model development and pilot periods. (See Table 5 for 


more information on the typical costs of lesson study.) Faculty “he igh ees 
charged with facilitating 

During the pilot study, teams met for between 15 and 18 hours the lesson study 

to complete all of the lesson study cycle activities. While this isa pilot cycle invested 

comparatively time-consuming professional development activity substantially more time 

in higher education, many faculty members appreciated the depth than their colleagues 

and intensity of their experience. As noted above, the teams were who engaged as 
participants. 


less likely to find time to identify and read research literature and 
disseminate knowledge gained from the lesson study cycle, two key 
components of the lesson study model. This may be because it would require additional 
time outside of the cycle meetings or expertise in content-specific pedagogy that faculty 
would need ongoing support to develop. This indicates that the time invested was not 
quite sufficient to successfully implement every component of lesson study. 


Faculty who were charged with facilitating the lesson study pilot cycle invested 
substantially more time than their colleagues who engaged as participants. Faculty 
facilitators took on a range of logistical and administrative tasks to make the cycle 
successful, including recruiting their colleagues to participate, securing meeting rooms, 
coordinating and aligning schedules for meetings and observations, and arranging for 
substitutes in cases where faculty needed to miss class to observe the teaching or reteaching 
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sessions. In addition, faculty facilitators indicated that they spent time reviewing the 
facilitation materials in preparation for cycle meetings. It is important to note that 
members of the faculty leadership teams who served as facilitators in the pilot cycle drew 
on the knowledge and experience that they gained during the model development phase. 
Institutions planning to implement lesson study will need to identify faculty facilitators 
with prior knowledge of lesson study or provide support for facilitators to pursue training. 


Faculty facilitators indicated that it would not be feasible to 


continue playing this role without either released time from Faculty facilitators 


teaching or other forms of support from their college. One faculty indicated that it would 
facilitator explained that she did not wish to continue coordinating not be feasible to 

lesson study in her department, but she would participate if continue playing this role 
someone else would lead the initiative. In one example ofa without either released 
sustainable approach to lesson study in higher education, the time from teaching or 
Center for Teaching and Learning at the University of Wisconsin at other forms of support 
LaCrosse has been administering a lesson study program for nearly from their college. 


20 years (Cerbin & Kopp, 2006). The program operates with center 

staff matching interested faculty, providing materials, and offering 

a modest stipend for participation. One of the three colleges in our study continued lesson 
study beyond the project period by using college funds to provide released time from 
teaching for faculty facilitators and pay part-time faculty for their participation. The other 
colleges intended to implement “light-touch” versions of lesson study that preserved 

the most important features of the model (i.e., classroom observation focused on student 
learning) but required less faculty coordination and time. Additional research is needed to 
understand if a less intensive model of lesson study can yield similar benefits for faculty. 


Table 5. 
Costs of Lesson Study 
Time for training facilitators 15-18 hours per team member 
Time for recruiting faculty and forming teams Additional 5-10 hours of facilitator preparation 
Time for polling faculty to identify meeting times, booking Payment for substitute instructors to cover team members’ 


meeting rooms, processing payments for part-time faculty, etc. | classes during scheduled meetings or observations (as needed) 


Findings on Student Outcomes 


To assess the relationship between faculty participation in lesson study and students’ 
learning, course grades, and progression into college-level math, we compared the 
outcomes of students in course sections taught by faculty who participated in the lesson 
study pilot to the outcomes of students in sections taught by nonparticipating faculty. 


Student Learning 


There was no existing common measure of student learning in the quantitative literacy 
pathway across the three colleges, so faculty leaders worked with the research team to 
develop an assessment that would test student knowledge ofan essential concept in the 
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course and provide meaningful information about student learning to the teams. The 
team created a four-item assessment focused on percentages—a key concept that underlies 
many skills and learning outcomes in the quantitative literacy pathway. 


Allassessment items required that students refer to a table of sample data: two years of 
enrollment trends ata community college in California, shown as the percent enrollment 
by age group. The items were open-ended, which is reflective of the kinds of tasks and 
real-world applications that students were typically expected to encounter during the 
course. It is important to note that this brief assessment did not cover all the mathematical 
topics addressed by the four lesson study pilot teams in their lesson study cycles, nor is it 

a comprehensive measure of student learning in the course. Only two of the four teams 
specifically focused their pilot lesson study cycles on the topic of percentages. 


The learning assessment was administered in the final two weeks of the fall 2019 
academic term in Math 058 and 098 sections taught by lesson study faculty participants 
and nonparticipants. A total of 197 students completed the assessment; 60% of those 
students were enrolled in a course section taught by a faculty member who participated in 
lesson study. A sample of 181 students who completed the assessment in fall 2018 served 
as a comparison group. Student assessments were submitted to CCRC for scoring using a 
rubric developed and tested in partnership with faculty. Students received one point for 
each correct answer. (See Appendix C for a copy of the assessment and scoring procedures 
as well as additional details on the analysis and results.) 


In Math 098, we did not see a statistically significant difference In Math 058, we saw a 

in performance between students taught by participating faculty positive, statistically 

and those taught by nonparticipating faculty. However, for the 111 significant effect of faculty 
students enrolled in Math 058 who completed the assessment, we lesson study participation 
saw a positive, statistically significant effect of faculty lesson study on student performance 


on the second and third 


participation on student performance on the second and third is 
assessment items. 


assessment items. The second and third items asked students how an 
increase in enrollment for one age group would affect that age group’s 
proportion of total enrollment. These questions required students to understand howa 
percentage reflects the relationship between a part anda whole. After accounting for Math 
058 students’ demographic characteristics and enrollment information, logistic regressions 
showed that students in sections taught by participating faculty were about twice as likely 
to answer the second item correctly and about three times as likely to answer the third item 
correctly compared to students in sections taught by nonparticipating faculty. 


Students’ Course Grades and Progression Into 
College-Level Math 


To assess differences in grades and progression between students enrolled in sections taught 
by lesson study participants and those enrolled in sections taught by nonparticipants, we 
analyzed student-level administrative data from the three colleges with a focus on four 
outcomes: their grade in Math 098, whether they passed the course with a C or better, their 
persistence to the end of the course (thatis, they did not withdraw or have an incomplete 
grade), and their enrollment in college-level math in the subsequent term (winter 2020). In 
this study, college-level math included college-level algebra and math for liberal arts. 
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Our sample includes students taught by 15 faculty members in 18 sections of Math 098 
in the fall 2019 term.‘ Eight of those faculty members teaching 10 sections participated in 
the lesson study pilot. We first examined the descriptive outcomes in the pilot term (see 
Table 6). Compared to students in sections taught by nonparticipants, students in sections 
taught by lesson study participants earned slightly lower grades and were less likely to 
pass Math 098, buta similar percentage of students persisted to the end of the course and 
progressed to college-level math. 


Table 6. 
Descriptive Course Outcomes of Students in Math 098 Sections Taught by Lesson Study 
Participants and Nonparticipants, Fall 2019 


LESSON STUDY NON-LESSON 
SECTIONS STUDY SECTIONS 


Number of Students 171 160 


Student Course Outcomes 


Math 098 letter grade 2.39 2.68 
Passed course (A, B, C, or “Pass”) 70% 81% 
Persisted to course end 96% 97% 
Progressed to college-level math 33% 32% 


Note. For the course outcomes, letter grades of 2.39 and 2.68 correspond toa C. 
Source. Authors’ analysis of community college administrative data. 


To account for differences in outcomes that could be due to student characteristics, 

we used propensity score weighting and regression analysis to assess the relationship 
between enrollment in a Math 098 course taught by faculty participating in lesson 
study and the outcomes of interest.° To identify the more direct impact of lesson 

study on student outcomes, the study would have had to ensure that students in the 
sections taught by lesson study participants and nonparticipants had similar academic 
achievement (as determined by a continuous academic measure) and socioeconomic 
status at baseline or prior to the start of the fall 2019 (What Works Clearinghouse, 
2019). This type of quasi-experimental design could have potentially met What Works 
Clearinghouse (W WC) standards with reservations and provided some indication of 
the impact of lesson study on students.° However, we did not have a measure of academic 
achievement prior to the start of fall 2019 for students in the sample. By constructing a 
matched comparison group, we were able to ensure that students taught by lesson study 
participants and those taught by nonparticipants were demographically equivalent. In 
addition, we used a multilevel model to account for which college the students attended, 
as well as for instructor- and classroom-level differences because we found substantial 
variation in the math course grades and progression outcomes by instructor.” 


After controlling for student characteristics, instructor, and college, we found that taking 
a Math 098 course with a faculty member who participated in lesson study had a negative 
association with passing the course (similar to the descriptive findings in Table 6). There 
was no association between taking a Math 098 course taught by a lesson study participant 
and grades in Math 098, persistence to the end of the course, or progression to college- 
level math. See Appendix D for further detail on sample characteristics, the analytic 
method, and results. 
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A Theory of Change for Lesson Study’s Effects on 
Students 


Although this project found several benefits for faculty participating in lesson study and 
stronger performance on some assessment items for some students, we did not finda 
positive relationship between faculty participation in lesson study and student course 
performance. In this section, we contextualize these findings within the limitations of 
our study design and propose a theory of change that can inform the design of future 
research on the impact of lesson study on student outcomes. 


The pilot study design was limited, as noted above, by the fact that we could not account 
for potential differences in prior academic achievement between students taught by 
participating and nonparticipating faculty. Additionally, a pilot study focusing on one 
term may not provide enough time to test the full effects of lesson study, as it may take 
additional time for faculty to incorporate what they learn from lesson study into their 
teaching practice. During the lesson study pilot cycle, faculty revised one lesson. The 
lesson study implementation practices suggest that faculty should extrapolate learnings 
from that single lesson to apply to their course more broadly. Just as previous research has 
shown that effective professional development is long-term and sustained (e.g., Darling- 
Hammond et al., 2017; Fowler et al., 2016), it is reasonable to expect that the broader 
rethinking and refinement of curricular materials and teaching practices involved in 
lesson study—and thus effects on students—would unfold over time. Finally, previous 
studies of professional development in higher education measure the impact on student 
learning outcomes aligned to the goals of the intervention (Condon et al., 2016; Stes et 
al., 2010). Our ability to extrapolate the effects of lesson study on student learning was 
hampered by our use of a short nonstandardized assessment that was not universally 
aligned with the focus of each team’s lesson study cycle. 


Based on our implementation research and drawing on Lewis and Perry’s (2015) lesson 
study model, we propose a theory of change that shows how lesson study may affect 
short-term outcomes ina single cycle and student outcomes after long-term faculty 
engagement (see Figure 6). This theory provides a framework for the design of future 
research on the impact of lesson study on instructors and students. 


As shown in the figure, this framework acknowledges that it takes more It takes more than one 
than one cycle for faculty to become comfortable with lesson study, cycle for faculty to : 
particularly given how different the approach is from other professional spied Seabee eaciteah 
development opportunities in higher education. A lesson study cycle sensu” on Ys particulany 
: . is given how different the 
is supported by an experienced facilitator and resources to address ‘ 

deni ; A lncistical ‘duende a nee approach is from other 
a ministrative and logistical consi erations. nce aculty ecome professional development 
familiar with the lesson study model, they increase their knowledge of opportunities. 
evidence-based practices, develop new understandings of students and 
their learning, form a productive professional community, and develop 
a refined lesson plan with each new cycle (short-term outcomes). The theory of change 
holds that after repeated cycles, these experiences will translate to improved instruction 
more broadly as faculty implement new teaching practices learned through lesson study 
throughout their courses. In the long term, improved teaching employing evidence-based 
instructional practices will lead to improved student learning and course performance. 
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Figure 6. 
Lesson Study Theory of Change for Effects on Student Outcomes 


During their first cycle, faculty After completing multiple In subsequent academic terms, after sustained 


become familiar with the lesson model lesson study cycles engagement with lesson study 


Inputs Short-Term Outcomes Intermediate Outcomes Long-Term Outcomes 
(Each cycle) (Each cycle) (Faculty) (Students) 


Instructor knowledge 


¢ Increase knowledge of 
evidence-based practices 
prudy, in their discipline 


/ & Plan \ ¢ Improve understanding of 
students’ learning 


Improved course 
performance 


Aon Teach, Improved . Students meet course 
- si ect ; Gheerve; Professional instruction learning objectives 
epor I ¢ Student i Ih 
P & Debrief community * Faculty draw on ae an cpt ¥ 
Develop and sustain a learning to refine mad i 
collaborative team with curricular : 
i Students experience 
" shared goals for materials and ‘ 
Revise higher course grades 


student learning teaching practices 
across the course 


and course completion 
rates, and enroll in 
subsequent courses in 
asimilar area of study 


& Reteach 


Refined practice 


¢ Develop a refined lesson 
¢ Experiment with new 
. ae evidence-based 
Supportive conditions instructional practices 
e Experienced lesson study facilitator 


e Administrative and financial support 


Improved learning 


¢ Students demonstrate 
improvements in 
learning on tasks tied to 
the refined lesson 


Based on this theory of change, future research should account for the time it takes for 
faculty to become comfortable with the lesson study approach and to then integrate new 
curricular and instructional practices into their course. To address the other limitations 
of the current study design, future research should account for baseline differences in 
student achievement and use an assessment of student learning that is either tied to the 
concepts taught in the cycle’s focal lesson or administered after faculty have engaged in 
multiple cycles. To account for variation in grading across instructors, future research 
should consider how faculty members grade before, during, and after their participation 
in lesson study to more accurately gauge lesson study’s effects on student course grades. 
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Conclusion 


A primary goal of this project was to understand if and how lesson study, an intensive 
and collaborative model of professional development used primarily in K-12 settings, 
could be adapted for use in community colleges. With input from mathematics faculty 
leaders at three community colleges, the project team developed a lesson study model 
that teams were able to implement with fidelity. Participants reported finding value 

in their lesson study experience, and our results indicate that they adopted new 
evidence-based instructional practices in mathematics. These findings are encouraging 
because there is limited research on models of professional development in higher 
education that result in changes to teaching. This study adds important knowledge 

to asmall but growing field of research. Moreover, research on mathematics teaching 
from K-12 and higher education shows that adopting cognitively demanding, student- 
centered approaches to instruction is challenging. Stigler and Hiebert (1999) highlight 
the deeply ingrained cultural habits associated with teaching that make it difficult 

to change and introduce new practices. Lesson study’s prescriptive protocols and 
structures help to facilitate in-depth conversations about teaching and learning. The 
experience experimenting with new teaching practices and looking closely at students 
and their learning helped faculty to increase their confidence with new instructional 
practices. As in K-12, lesson study in community colleges appears to be a promising 
model for supporting changes to teaching practice. 


While we were not able to detect consistent positive effects on student outcomes, likely 
due at least in part to the limitations of our study design, we drew on our implementation 
research to develop a theory of change for structuring lesson study to improve student 
outcomes. This project offers guidance for designing future studies of the impact of lesson 
study on students. 


Our implementation findings, along with other research, point to When a continuous 
the cultural and structural features of community colleges that make improvement culture is 
it challenging to scale up and sustain instructional improvement strong, the reform goals 


are clear to everyone at 


efforts (Demir et al., 2012). These challenges include a tendency 
the school. 


toward faculty autonomy and individualism, limited instructional 
resources, high teaching loads, and large numbers of part-time 
faculty. By contrast, the conditions in many K-12 schools—including collective or shared 
leadership focused on how to structure and support student learning, dedicated time 
throughout the year for teachers to collaborate on improving instruction, and regular use 
of data and cycles of inquiry to drive change (Bryk et al., 2010; Lytle, 2012; Mourshed et 
al., 2010; Robinson, 2011)—may be better able to support the implementation of lesson 
study. Under these conditions, K-12 schools are often committed to developing and 
strengthening a coherent curriculum across classrooms, and professional development is 
aligned with the specific improvement goals identified by the school. When a continuous 
improvement culture is strong, the reform goals are clear to everyone at the school. They 
are supported at multiple levels throughout the school, and all staff are more likely to hold 
themselves accountable for achieving them. 
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Without these conditions, lesson study may still meet the learning and improvement 
needs of individual faculty members who participate in the cycles. But if lesson study 
were to be integrated into a broader, goal-oriented, leadership-supported instructional 
improvement initiative, it may have the potential to generate more meaningful 
widespread change. As noted above, Centers for Teaching and Learning may offer one 
structural home for lesson study in higher education, in that they could provide the 
necessary logistical support that would make lesson study sustainable at a college. In 
our project, one college used lesson study as part of its annual departmental learning 
outcomes assessment. Connecting lesson study to existing initiatives could provide 
direction and focus to faculty teams, assist with challenges around disseminating and 
sharing knowledge, and support the sustainability of the model. 


In community colleges and in higher education more broadly, the 


As colleges rightly 
need to support faculty to improve instruction has never been focus thelr attention 
more urgent. Institutions are rapidly redesigning course structures on inequitable student 
and pathways, providing students referred to developmental outcomes by race/ 
education with accelerated access to college-level courses, ethnicity, they must 
and creating coherent program maps for all students. And in consider how to 
response to the COVID-19 crisis of 2020, colleges are increasing implement instructional 
their online course offerings, presenting unique challenges for approaches that could 
faculty. As colleges rightly focus their attention on inequitable close opportunity gaps. 


student outcomes by race/ethnicity, they must consider how to 

implement instructional approaches that could close opportunity 

gaps. The adapted lesson study model developed in this project provides an approach to 
supporting faculty to learn about evidence-based practices, examine student learning, 
and experiment with new instructional approaches. Institutional investment in these 
kinds of learning opportunities for faculty may support the kinds of transformations to 
instruction that are needed in the current higher education context. 


Endnotes 


1. More information on the adapted model, including facilitation materials, can be found 
in EdNW (2021). 

2. Seven of the 22 survey respondents participated in only the pilot cycle. The 
remaining respondents participated in at least one additional cycle during the model 
development phase. 

3. Items drawn from the Organisation for Economic Co-operation and Development’s 
(2018) Teaching and Learning International Survey. 

4. Weexcluded Math 058 students from this analysis because this course does not lead 
directly to college-level math. 

5. We originally intended to use an interrupted time series design for this analysis; 
however, this design was not appropriate to use since only about half of the Math 098 
faculty members participated in lesson study in fall 2019, and the design would have 
pooled outcomes across lesson study and non-lesson study sections in fall 2019 (see 
Appendix D for more details). 
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6. WWCusesaset of standards and procedures to review education research in 
a consistent and objective manner and to answer the question: What works in 
education? 

7. We found substantial variation in the course grades and progression outcomes by 
instructor. For example, pass rates in Math 098 ranged from 42% to 91%, and course 
persistence rates ranged from 64% to 100%. 
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